Haptics for the development of fundamental rhythm skills, including multi-limb coordination

This chapter considers the use of haptics for learning fundamental rhythm skills, including skills that depend on multi-limb coordination. Different sensory modalities have different strengths and weaknesses for the development of skills related to rhythm. For example, vision has low temporal resolution and performs poorly for tracking rhythms in real-time, whereas hearing is highly accurate. However, in the case of multi-limbed rhythms, neither hearing nor sight are particularly well suited to communicating exactly which limb does what and when, or how the limbs coordinate. By contrast, haptics can work especially well in this area, by applying haptic signals independently to each limb. We review relevant theories, including embodied interaction and biological entrainment. We present a range of applications of the Haptic Bracelets, which are computer-controlled wireless vibrotactile devices, one attached to each wrist and ankle. Haptic pulses are used to guide users in playing rhythmic patterns that require multi-limb coordination. One immediate aim of the system is to support the development of practical rhythm skills and multi-limb coordination. A longer-term goal is to aid the development of a wider range of fundamental rhythm skills including recognising, identifying, memorising, retaining, analysing, reproducing, coordinating, modifying and creating rhythms – particularly multi-stream (i.e. polyphonic) rhythmic sequences. Empirical results are presented. We reflect on related work, and discuss design issues for using haptics to support rhythm skills. Skills of this kind are essential not just to drummers and percussionists but also to keyboards players, and more generally to all musicians who need a firm grasp of rhythm

Haptics in the Air - Exploring Vibrotactile Feedback for Digital Musical Instruments with Open Air Controllers

When playing a traditional musical instrument, the performer often relies on vibration that is produced by the instrument. When perceived through tactile sensing, this can be defined as \emph{vibrotactile feedback}. Since sound in digital musical instruments (DMIs) is not produced by mechanical vibration of its constituent parts, vibrotactile feedback is inherently absent. This means that DMIs are lacking an important feedback modality. DMIs can be played using a wide range of different controllers. \emph{Open air} controllers can make use of motion carried out in open air to control sound. These controllers are particularly prone to the issues related the lack of vibrotactile feedback since they may not have a tangible interface. In this thesis it was investigated how open air controllers can be augmented with vibrotactile feedback. With basis in relevant theory and previous attempts, two DMI prototypes based on open air control of sound were developed. The prototypes allowed control of musical sound on a high and low level. Open air motion was captured using motion capture technology. In this case, the control surface consisted of a tangible element, such that actuators could be embedded in the controller. It was investigated how vibrotactile feedback can convey musical information. This issue was investigated from both a theoretical and practical approach. The practical approach entailed providing vibrotactile feedback to the fingertips of the performer using signals that were synthesized in musical programming environments. Preliminary results of an informal evaluation of the developed vibrotactile feedback strategies suggest that information on musical parameters such as amplitude and timbre can be conveyed with vibrotactile feedback. While the importance of vibrotactile feedback is stressed in the literature, the preliminary results also show that the developed feedback strategies can be found useful